Purpose

This document lays out results for estimating statistical power and minimum detectable effect size (MDES) for: Blocked RCT, with 3 levels, and randomization done at level 1 (individual level). This document contains models assuming constant treatment effects, fixed treatment effects, and random treatment effects.

We compare results of the derived methods against Monte Carlo Simulations & the PowerUp R package on power. The methods are derived by Kristin Porter as outlined in the paper/s here (Add reference section).

Power Validation

In this section, we validate power results for different definitions of power and different adjustment procedures.

If you are previewing this in html, please click on the code section to review more detailed parameters.

This section sets up the simulation-level parameters, such as how many monte carlo samples to draw.

Setup 1: School size 100

Main parameters:

  • n.j = 100
  • M = 3
  • J = 20
  • K = 4
  • rho = (0.5, 0.5, 0.5)
  • MDES = (0.125, 0.125, 0.125)
  • R2.1 = (0, 0, 0)
  • R2.2 = (0, 0, 0)
  • R2.3 = (0, 0, 0)
  • ICC.2 = (0, 0, 0)
  • ICC.3 = (0, 0, 0)
  • Omega.2 = 0.5
  • Omega.3 = 0.5

Individual Statistical power is estimated to be around 0.8. Please check the table below for estimations of other definitions of statistical power. Across all power definitions, estimation results across the PUMP package, Monte Carlo Simulation results and PowerUp package are about the same as detailed below.

Setup 2: School size 75

  • n.j = 75

Setup 3: School size 50

  • n.j = 50

Varying R2

R2.1 = 0.6, 0.6, 0.6

R2.2 = 0.6, 0.6, 0.6

R2.3 = 0.6, 0.6, 0.6

Varying rho

rho.default = 0.2

rho.default = 0.8

Varying true positives

MDES = 0.125, 0, 0

Varying ICC

ICC.2 = c(0.8, 0.8, 0.8)

ICC.3 = c(0.8, 0.8, 0.8)

Varying Omega

omega.2 = 0.8, omega.3 = 0.5

omega.2 = 0.5, omega.3 = 0.8

omega.2 = 0.8, omega.3 = 0.8